A Dual-System Approach to Realistic Evaluation of Large-scale Networked Systems

A Dual-System Approach to Realistic Evaluation of Large-Scale Networked Systems Richard Allan Alimi 2010 Many useful techniques and infrastructures have been developed to test the correctness and performance of computer systems before deployment. However, they are limited when applied to the large-scale networked systems that are prevalent today, from Internet routing to peer-to-peer (P2P) video delivery. First, testing infrastructures are typically smaller in size and scale than the production infrastructure, limiting the scale of possible test scenarios. Second, keeping the testing infrastructures and their configurations synchronized with the production infrastructures is difficult or impossible. Finally, capturing and integrating production workloads into the testing infrastructures can be challenging. To address these shortcomings, we have developed novel techniques to conduct correctness and performance evaluations on production infrastructures for large, networked systems. We show how incorporation of domain-specific knowledge of a system being tested can provide an efficient testing infrastructure that is scalable, accurate, and easily integrates production workloads. Our techniques also avoid disruption to users of the production system. In particular, we first present a general formulation of a dual system running both the production and the tested systems concurrently. We then present ShadowNet, a realization of the dual-system concept at the link layer and network layer. ShadowNet can be used by network providers (e.g., ISPs) to evaluate the correctness and performance of network configurations before deployment. We also present PEAC, a realization of the dual-system concept at the application layer, to introduce additional capabilities. In particular, using P2P live streaming as an example, we show how the tested system can be used to handle the production system’s workload while avoiding disruption to users. PEAC introduces a scalable and distributed control technique that allows a developer to construct test scenarios with a large number of nodes.